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1 Introduction 



In this paper, we consider the singular stochastic control problem for systems 
governed by nonlinear controlled diffusion of the type 

dxt = f {t, xt, ut) dt + a {t, xt, ut) dWt + Gtdr]^, ,^ ^, 

where {Wt)t is a standard /— dimentional Brownian motion defined on the 
filtered probability space {^,F, {Ft)^ ,P). The minimized criteria associated 
with the state equation p.ip is defined by 



(xt)+ [ iit,xt,ut)dt+ I , 

J S J S 



hdr]t 



(1.2) 



where E denotes the expectation with respect to P, and the value function 
is defined as 

V{s,y)= inf {J(s,y,u,r/)}. (1.3) 

(M,r,)GU([s,T]) 

The kind of stochastic control problem has been investigated extensively, 
both by the Bellman's dynamic programming method [6] and by Pontrya- 
gin's maximum principle |18] . In this paper, we are concerned by th second 
method. Peng |T7] introduced the second-order adjoint equation and ob- 
tained the global maximum principle of optimality, in which the control is 
present in the both drift and diffusion coefficients. Studying near-optimal 
controls makes a good sense as studying optimal controls from both the- 
oretical as well as applications point of view. Many more near-optimal 
controls are available than optimal ones, indeed, optimal controls my not 
even exist in many situations, while near-optimal controls always exist. The 
near-optimal deterministic controls problem has been treated by many au- 
thors, including [I2l [20l EH [15]. Recently, in an interesting paper, Zhou 
|22j established the second-order necessary as well as sufficient conditions 
for near-optimal stochastic controls for general controlled diffusion with two 
adjoint processes. The near-optimal control problem for systems descripted 
by volterra integral equations has been studied by [16J. However, Chighoub 
et al., [8] extended Zhou's maximum principle of near-optimality to SDEs 
with jumps. The similar problem for systems driven by forward-backward 
stochastic differential equations has been solved in [5]. For justification of 
establishing a theory of near-optimal controls see ( \22\ [20] , Introduction) . 

Singular stochastic control problem is an important and challenging class 
of problems in control theory, it appear in various fields like mathematical 
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finance, problem of optimal consumption etc. Stochastic maximum principle 
for singular controls was considered by many authors, see for instance [H 
El 13 ITOl [TTl [3l [T3l [13] • The first version of maximum principle for singular 
stochastic control problems was obtained by Cadenillas et al., |7|. The first- 
order weak stochastic maximum principle has been studied in [3]. In [TU] the 
authors derived stochastic maximum principle where the singular part has 
a linear form. Sufficient conditions for existence of optimal singular control 
have been obtained in [llj . 

The main objective of this paper is to establish necessary as well as suf- 
ficient conditions for near-optimal singular control for SDEs. The control 
domain is not necessarily convex. The proof of our result is based on Eke- 
land's variational principle [12], and some delicate estimates of the state and 
adjoint processes. Finally, as an illustration an example is solved explicitly. 
This result permit us to extend Zhou's maximum principle of near-optimality 
to singular control problem. 

The paper is organized as follows. The assumptions and statment of the 
control problem is given in the second section. In the third and forth section, 
we establish the main result of this paper. 



2 Assumptions and statement of the problem 

We consider stochastic optimal control of the following kind. Let T be a 
fixed strictly positive real number and (0,, F, {Ft}^ ,¥) be a fixed filtered 
probability space satisfying the usual conditions in which a /— dimentional 
Brownian motion W = {Wt '■ s <t < T} with s G [0, T] and Ws = is 
defined. Let Ai be a closed convex subset of and A2 := ([0, 00))™ . Let 
Ui be the class of measurable adapted processes n : [s, T] x — )• Ai and ILJ2 
is the class of measurable adapted processes r] : [0, T] x ^ A2. 
Definition 1. An admissible control is a pair {u,r]) of measurable Ai x A2- 
valued, Ff— adapted processes, such that 

1) 7] is of bounded variation, nondecreasing continuous on the left with right 
limits and rj^ = 0. 

2) E supt^[g^T] \utf + |r/5n|2 < 00. 

We denote U = Ui x U2, the set of all admissible controls. Since drj^ may be 
singular with respect to Lebesgue measure dt, we call t] the singular part of 
the control and the process u its absolutely continuous part. 
Throughout this paper, we also assume that 

(HI) / : [0,r] X M^xAi^M", a : [0,T] x WxAi^MnxiO^) and £ : 
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[0, T] xM" X Ai —7- M, are measurable in (t, x, u, ) and twice continuously 
differentiable in x, and there exists a constant C > such that, for 
ip = f,a,i: 

\ip{t, x,u, ) — ip{t, x' , U,)\ + \iPx{t, x,u,) — (p,j.{t, x' ,u,)\ < C \x — x'\ . 

(2.1) 

\^{t,x,u,)\<C{l + \x\). (2.2) 

(H2) h : M is twice continuously differentiable in x, and there exists 

a constant C > such that 

\h{x) - h{x'))\ + |/i^(x) - h^{x'))\ <C\x-x'\. (2.3) 

\h{x)\ < C {I + \x\) . (2.4) 

(H3) G : [0,T] ^ 7W„xm (M) , A: : [0,r] ^ ([0,oo))'" , for each t £ [0,T] : G 
is continuous and bounded, and k is continuous. 

Under the above assumptions, the SDE (II. ip has a unique strong solution 
xt which is given by 

Xt = y+ I f {r,Xr,Ur) dr + / a {r,Xr,Ur) dWr + / Grdr]^, 
Js Js Js 

and by standard arguments it is easy to show that for any g > 0, it hold 
that 



E( sup Ixtl") <C{q), 
te[s,T] 

where C (q) is a constant depending only on q and the functional J is well 
defined. 

For any {u, r/) S U and the corresponding state trajectory x, we define the 
first-order adjoint process and the second-order adjoint process Qt as the 
ones satisfying the following two backward SDEs respectively 



d-^t = - [fx {t, xt,ut) "^t + cr*x {t, xt,ut) Kt + 4 {t, xt,Ut)] dt 

+ KtdWu (2-5) 
= hx (xt) , 
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and 



dQt = - [fx xt,ut) Qt + Qtfx (t^ ^t, ut) + K (t, xt,ut) Qtcr* {t, xt, Ut) 
+a* {t, Xt, Ut) Rt + RtCTx {t, xt,ut) + Tj] dt + RtdWt, 
= hxx (xt) , 

(2.6) 

where 

^ . . 

Tt = ixx (t, Xt, Ut) + E mfxx Xt, Ut) + Klal^ {t, Xt, Ut)) . 
1=1 

As is well known, under conditions (HI), (H2) and (H3) the first-order 
adjoint equation (|2.5p admits one and only one F— adapted solution pair 
(^,Er) G L2,([0,r] ,M")xL2,([0,r] ,M") and the second-order adjoint equa- 
tion (12. 6p admits one and only one F— adapted solution pair {Q, R) G 
hi ([0,r] ,R"X") X L| ([0,T] ,M"X") . Moreover, since fx,ax,ix and hx are 
bounded then we have the following estimate 



E 



sup \^t\^+ / \Ktfdt+ sup \Qtf+ / \Rt\^ 

s<t<T Js s<t<T Js 



< C. 



Define the usual Hamiltonian 

H {t, X, u,p, q) := —pf (t, x, u) — qa {t, x,u) — i {t, x, u) , (2.7) 

for {t,x,u) € [s,T] X M" x Ai. Furthermore, we define the Ti functional 
corresponding to a given admissible pair (x, u) as follows 

'H^''^''\t,x,u) = H{t,x,u,^t,Kt-Qta{t,x,u)) 

— -O"* (t, X, u) QtCT {t, X, u) , 

for {t,x,u,p,q) G [s,T] x x AixM" x M", where Kt and Qt are 
determined by adjoint equations ()2.5p and ()2.6p corresponding to {x,u). 
Befor concluding this section, let us recall Ekeland variational principle and 
the Clarke's generalised gradient as follows 

Lemma 1. (Ekeland's Lemma 112^ ) Let {F, p) be a complete metric space 
and f : F —^M be a lower semi- continuous function which is bounded below. 
For a given e > 0, suppose that u^ F satisfying f {u^) < inf (/) + £, then 
for any A > 0, there exists u^ ^ F such that 

1) f{u>^)<f{un. 

2) p{u^,u^) < A. 
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3) f (n^) < f{u) + jp {u, u^) for all u e F. 

For n, t> E U To apply Ekeland's variational principle to our problem, we 
define a distance function p on the space of admissible controls such that 
(U, d) becomes a complete metric space. To achieve this goal, we define for 
any {u,t]) and (^,0 E U : 

d{{u, ri) , {v, 0) = di (n, v) + ^2 (r?, , (2-8) 

where 

di {u,v) =¥(g)dt{{w,t) enx [0,r] :u{w,t) ^v{w,t)}, (2.9) 

and 

d2 



E( sup 1% -C/) 



(2.10) 



tG[s,T] 

here P(8>(it is the product measure of P with the Lebesgue measure dt on 
[s, T] . It is easy to see that (U2, £^2) is a complete metric space. Moreover, it 
has been shown in Yong et ai, ([19] pp. 146-147) that (Ui, di) is a complete 
metric space. Hence (U, d) as a product of two complete metric spaces is a 
complete metric space under d. 

Definition 1. (Clarke's generalized gradient [9j) Let E he a convex set 
in M" and let / : — > M be a locally Lipschitz function. The Clarke's 
generalized gradient of / at x G denoted by 9^^/, is a set defined by 

dxf (x) = U e : v) < lim sup / + ~ / (^) ^ Vt; G M and y, {y + tv) G e] 
[ v-'^t^o t J 

3 Necessary conditions for near-optimal singular 
control 

Our purpose in this paper is to establish second-order necessary and suffi- 
cient conditions for near-optimal singular control for systems governed by 
nonlinear SDEs. It is worth montioning that optimal singular controls may 
not even exist in many situations, while near-optimal singular controls al- 
ways exists. Ekeland's variational principle [12] is applied to prove our 
maximum principle. The proof follows the general ideas as in ( |2m ETj 122] ) 
where similar results are obtained for other class of controls. 
We give the definition of near-optimal control as given in Zhou ([22], Defi- 
nition 2.1 and Definition 2.2). 
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Definition 2. For a given e > the admissible control {u^,r]^) is near- 
optimal if 

\J{s,y,u',v')-V{s,y)\<0{e), (3.1) 

where O (.) is a function of e satisfying lime_>>o O (e) = 0. The estimater 
O (e) is called an error bound. If O (e) = Cs^ for some 6 > independent 
of the constant C then [u^^r]'^) is called near-optimal control with order . 
If O {e) = e the admissible control (u^,t/'^) called e— optimal. 
Our first Lemma below, is deals with the continiuity of the state processes 
under distance p 

Lemma 2. If x^'^ and x^'^ be the solution of the state equation U.l\) 
associated respectively with u and v. For any < a < 1 and /3 > satisfying 
a/3 < 1, there exists a positive constants C = C {T,a,l3) such that 



E( sup 

s<t<T 



u,r] 



2/3 



) < Cdf {u,v). 



(3.2) 



Proof. First, we assume that f3 > 1. Using Burkholder-Davis-Gundy in- 
equality for the martingale part, we can compute, for any r > s 



E 





u,r] v,^ 


2/3" 


sup 




_s<t<r 







< CE 
+ 



/ {t,Xt''^,ut) - f (^t,X^'^,Vt^ 
a {t,x^''^,ut) - a {t,xl'^,vi^ 



2/3 



dt 



+CE |r/r - 
< CE 

+ 



2/3 



( [\f{t,xl\ut)-f{t,x't^\vt)f 

a{t,Xt''^,ut) - o-(t,x"''',wt)|^'^| Xut^vt it)dt 
+CE( r {\f{t,x'l'\vt)-fit,xt'\vt)t 



1 2/3 



+ 



a{t,x^'^,vt) - a{t,Xt-'^,vt 



1 2/3 



-fCE \riT - 



2/3 



now arguing as in (|22]. Lemma 3.1) taking b = > 1 and a > 1 such that 



1 I 1 



al3 

1, and applying Cauchy-Schwarz inequality, we get 



7 



J s 

using definition of di and linear growth condition on / we obtain 



E \f{t,x'i^\ut)-f{t,x'i'\vt)fxu.^.At)dt 

< C |e (l + \xl''f'') dtj " di {u, 

< Cdi{u,v)"^. 
Similarly, we can prove 

E ^ V (t, ut)-a {t, xl\ vt)f Xu,+., (t) dt < Cd, (n, v^^ . (3.3) 

Therefore, by using assumption (HI), we conclued that 



E( sup 

s<t<r 



u,ri v,^ 



2/3, 



) <C {E I sup 

Is s<r<e 



u.ri 



2/3 



de + E\r]rp-^j,\'^^ + di {u,v 



Hence (jS.ip follows immediately from definition 1 and Gronwall's inequality. 
Now we assume < /3 < 1. Since § > 1 then the Cauchy-Schwarz inequality 
yields 



E( sup 

s<t<T 



u,ri v,(^ 



2/3, 



) < 



E( sup 

s<t<T 



u,ri v,S, 



/3 



< [cdi{u,vrf 

< Cdi{u,v)''^ . 
This completes the proof of Lemma 2. 

Lemma 3. For any < a < 1 and 1 < /3 < 2 satisfying {1 + a) (3 < 2, 
there exist a positive constant C = C {a, 13) such that for any {u,r]), (f,C) ^ 
U([s,r]), along with the correspending trajectories x"'^, x""'^ and the solu- 
tions {"^jKjQjR), , K' ,Q' , R') of the corresponding adjoint equations, it 
holds that 
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E {\^t-m +\Kt- KT)dt < Cd^^ {u,v). (3.4) 





-<+ 


\Kt 




'0 
















[ {\Qt 


\Rt 


-R't\ 


Jo 









i\Qt - Q't\'" + \Rt - R'tf)dt < Cdf {u,v). (3.5) 

JO 

Proof. Since the adjoint processes are independant to singular part, we use 
similar argument as in Zhou ([22] Lemma 3.2). 

Now we are able to state and prove the necessary conditions for near- 
optimal singular control for our problem, which is the main result in this 
paper. 

Let (^'^,i('^) and {Q^,R^) be the solution of adjoint equations h2. 5|) and 
Ii2. 6\) respectively corresponding to {x^ , {u'^ ,r]^)) . 

Theorem 1. (Maximum principle for any near-optimal singular control). 
For any 5 € (0, |], and any near-optimal singular control (u^, ry^) there exists 
a positive constant C = C (5) such that for each e > 

' -Ce^ < E/J {i (a (t, xlu) - a {t, xf, uf ))* Qf {a (t, x',u) - a {t, xf, nf )) 

(/ (t, xlu) - f it, xl nf )) + {a (t, xlu) - a (t, xf, uf )) 

+ {i {t,xf,u)-i{t,xf,ul))}dt, 

(3.6) 

and 



[ {kt + G:^)d{r,-r,^\ 
Jo 



(3.7) 



2 

Proof. By using Ekeland's variational principle with A = £3, there is an 
admissible pair (x^, (u^,r/^)) such that for any (n, ?]) € U : 

p{{u',r^'),{lf,rj'))<el (3.8) 

and 

J {s,y,u^,rj^) < J {s,y,u'',rj^) e^d((u,r]) ,{u^,r]^)) . 

Notice that {u^,r]^) which is near-optimal for the initial cost J defined in 
(II. 2p is optimal for the new cost given by 

{s,y,u,7]) = J {s,y,u,r]) + e^d{{u,r]) , (?f ,7?^)) . (3.9) 

then we have 

J^s,y,lf,r]'')<J''{s,y,u,rj) for any (u, r/) G U ([s, T]) , 
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Next, we use the spike variation techniques for n'^to drive the first variational 
inequahty and we use convex perturbation for rj^ as follows 
First variational inequality: For any 9 > 0, we define the following 
strong perturbation {ul'^ ,fjf) € U : 

(Tf'.nD^i + (3.10) 

I [uf,r]f) , otherwise. 



The fact that 
and 

imply that 



r{s,y,lf,v') < r{s,y,lf'',rf), (3.11) 
d{{jf'\rft),{vr',rft)) = di{vr''\tf)<e, 



Since the diference J(s, y, u^'^, r/^)) — J {s,y,u'^ ,fj'^) is independant to the 
singular part, the near-maximum condition (|3.6p follows by applying similar 
argument as in Zhou ([22J), we get 

-Ce§ < E/J {i (a (t,xf,n) - a (t,xf ,nf))* (a (^,x^n) - a (t,xf ,IIf)) 
+ if {t, xt,u)-f {t, xf,uf)) + Kl {a {t, xlu)-a {t, xf,uf)) 
+ {£ {t,xl,u)-£{t,xlul))}dt. 

(3.12) 

New we are to derive an estimate for the term similar to the right hand side 
of the abov inquality with all the (xf , (uf , ryf)) etc, replacing by {xf, (uf , r]f)) 
etc. To this end, we use similar method as in Zhou ([22j) we obtian the 
following estimates: 

E/f [Klia{t,xlu)-a{t,xt,ul)) 
- KI{a{t,xlu)-a{t,xl,ul))]dt<Ce^, 

and 



^ll {h Xt,u)-a (t, xf , uf ))* Qt (cr {t, x^,u)-a {t, xf, itf )) 

-i (o- (t, xl,u) - a {t, xf,ul)y Ql {a {t, x'',u) - a [t, xf , )) 

+ [W, if (t, xf, u)-f {t, xf , Hf )) - (/ (t, xf , n) - / {t, xf , nf ))] (3.14) 

+ [£{t,xlu)-eit,xt,ul)] - [£ {t,xlu)-£{t,xlumdt 

< Ce\ 
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where ["^ ,K ) and [Q ,R ) are the solutions of adjoint equations (|2.5p 
and (j2.6p respectively corresponding to (x^, (u^, rj^)) . The first variational 
inequality (j3.6p follows from combining, (|3.12p (|3.13p and. p.l4p 
Corollary 1. Under the assumptions of Theorem 1, we have 

/•T fT 

E/ n^''''''"\t,xlut)dt> sup E/ ?^(^''"')(t,xf,nt)dt-C7e^ 

Js ueV{[s,T]) Js 

(3.15) 

Second variational inequality: To obtain the second variational inequal- 
ity, we define the following convex perturbation {uf,r]f^) G Ui x U2 : 



{ut,rf/) = ilf„r]t + e{C-r]t)), (3.16) 

where ^ is an arbitrary element of the set U2. Using the optimality of (u^,rj^) 
to the new cost, we have 

J^(s,y,n^r) < r{s,y,ulrjf), (3.17) 

a simple computation on d2{uf,r)f' ) we obtain 

J{s,y,Tti,ff/)-J{s,y,u',ff) > -Cfei > -COe^. 

Finally, arguing as in (|4j) for the left-hand side of the above inequality, then 
we have 
1 



lim 

6*^0 



J{s,y,ul,ri^' ) - J{s,y,u^,fj^) =E/ {h + Gl^t)d{ri -rf) 



It 1 



then the near-singular maximum condition follows 

l-T 



Ce" < E 



(fci + G[^';)d(r,-7?^ 



(3.18) 



Now, we are to drive an estimate for the term similar to the right hand side 
of p.lSp with all (x^, (u^, r/^)) ect., replacing by (x^, (n^, r/^)) ect. We first 
estimate the following difference: 



E fih + GrM/^)d(77 - r)t - E f{h + Gm)d{r^ " ri')t 

J s J s 

E r G; {W^ - ^) dvt + E + Gt^)drjt 

T 



E / {h + Gl%)dr^l 



E 



/ Gl - d (r? - t)t + IE / {kt + G^^f )d (??^ - t)t ■ 

J S J S 
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Using the boundness of Gt, h, LemmaS, definition 1 (t/^ = = vl = 
we have 



E \r]rp — r/l-p + E \r]j, — rjj^\'^ < oo) and the fact that E (sups<t<T P ) ^ ^ 



E r{kt + GI^MV -rf)t-^ fih + Gm)d{ri - v')t < Ce' . (3.19) 

J s J s 

Combining (|3.18p and (|3.19p the proof of inequaUty ()3.7p is complete. 
Corollary 2. Under the assumptions of Theorem 1, we have 



E 



/ {h + Gin)dvl < , mf E / {h + Gm)dvt + Ge' . (3.20) 

4 Sufficient near- opt imality conditions 

In this section, we will prove that under an additional assumptions, the 
near-maximum condition on the Hamiltonian function is sufficient for near- 
optimality. We assume: 

(H4) p is differentiable in u for if = f,a,i and there is a constante C such 
that 



\ip{t,x,u,) - (p{t,x,u',)\ + \ip^{t,x,u,) - ip^{t,x,u',)\ < C 



u — u \ . 



(4.1) 



Theorem 2. Assume the H (t,-,-,'^f,K^) is concave for a.e. t E [s,T], 
P — a.s, and h is convex. Let (^'f,i^f), {Qf,R^) be the solution of the 
adjoint equation i2.5\) - l[KB\) associated with (n^,r/^) . If for some e > and 
for any (u, ?]) € U : 

/•T fT 

E/ n^''"'''"\t,xlut)dt> sup E/ ?^(^''"')(t,x?,nt)dt-e, (4.2) 

Js mGUi([s,T]) Js 



and 

E 

then we have 



ktd {ri - r] 



> -C£2, (4.3) 



J{s,y,u'',r]'') < inf J (s, y, n, 77) + Cea , (4.4) 

(M,r;)gU([s,T]) 

where C is a positive constant independent of e. 
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Proof. First, define the cost functional 

J (s, y, M, rj) = Ji (s, y, u) + J2 (s, 7/) , 

where 



(4.5) 



Ji (s,y,n) = E 



h [xt) + 



i{t,Xt,Ut) 



dt 



and 



J2(s,r?) =E 



Let us fix e > 0, Define a new metric d on U([s,T]) as follows: for any 
{u, rf) and (u, ^) € U : 



d ((m, 7?) , {v, i)) = di (n, v) + (i2 (r?, , 



where 



and 



di {u, w) = E 



?f = l + lvI/f| + |i^f| + |Qf| + |i2f|>l. 



(4.6) 



(4.7) 



Obviously di is a metric on (Ui , di ) , and it is a complete metric as a weighted 
norm. Hence (U, d) as aj)roduct of two complete metric spaces is a 
complete metric space under d. 
Define a functional T on Ui ([s, T]) by 

T(n) = E / H{t,x',u,^',K',Q')dt 

J s 
i-T 

= Ej ?^(^^«') {t,x',u)dt, 



a simple computation shows that 



|T(n) -T(7;)| < C7E 



?i \ut -vt\dt 



which implies that T is continuous on Ui ([s,T]) with respct to di. Now by 
using ()4.2p and Ekeland variational principle, there exists a € Ui ([s,T]) 
such that 



di{u^',u'^) < £2. 
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and 



where 



e[ n{t,xlM)dt= max E /" n(t,xf,Ut)dt, 

Js u&i{[s,T]) Js 



(4.8) 



(4.9) 



The maximum condition (j4.8p imphes a pointwise maximum condition namely, 
for a.e. t G [s, T] and P — a.s, 

?^(i,a;f,nf) = max7^(t, x^, u). 

Using Proposition Al (Appendix), then we have 

du'H{t,xt,lft) 3 0. 

Since |n — ufl is not differentiable in uf (locally Lipschitz), then we use 
Proposition Al (Appendix) we get 



By using (j4.9p and fact that the Clarke's generalized gradient of the sum of 
two functions is contained in the sum of the Clarke's generalized gradient of 
the two functions, we get 



dun{t,xt,ut) c a„?^("''"')(t,4,nf) + 



Applying the similar method as in ([22j) for the rest of the proof, we obtian, 
for an arbitrary u 



Ji (s, y, u^) < Ji {s, y, u) + Ce2 . 
Now, by using (|4.3|) we get 

rT 



(4.10) 



J2 (s,r?^ 



= E 
< E 



+ Ce-2, 



which implies that for an arbitrary r] 



(4.11) 
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Combining (|4.10p . (j4.1ip and (j4.5p we arrive at 

J {s,y,u'' ,7]") < J {s,y,u,r]) + Cei 
Since {u, r]) is arbitrary, the desired result follows. 

Corollary 3. Let the assumptions of Theorem 2 holds. A sufficient condi- 
tions for an admissible pair {x^ ,r]'^) to be e— optimal is 



E 



> 



n^''''''"\t,xlu'i)dt- / hdrj] 

J s 

sup e| rn^^'^^'\t,xt,ut)dt- r ktdA - 

M,??)eU([s,T]) VJs J s ) 



Example 1. Consider the one-dimensional stochastic control problem: n 
l = l,Gt = l,Ai = [0, 1], A2 = [0, l],m = 1, 



dxt = UfdWt + dr]^, 
xo = 0, 



and the cost functional being 



utdt + / kfdr] 



(4.12) 



(4.13) 



For a given admissible pair (x^ , {u^ ,r]^)), the correspending second-order 
adjoint equation is 

Qi — 1) 

By the uniqueness of this solution, (Q"^, i?^) = (1,0), then for any admissible 
control (n, r]) we have 



n^-''^'\t,xt,ut) = ut-{Kf-Qtut)ut-'^Qtul 

Replacing ut = uf, we get 

?^(^''"')(t,xf,nf) = 

= {[{utf + 2ut{l-Kf[ 
Hence by simple computation shows that if 

ut-{K!-l)e[0,l], (4.15) 



(nf - {K! - l)f - {Kt - if 
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then ([313]) and ([3:20]) gives 

"1 



E 







{utf + 2ul (1 - Kl) 
1 



dt 



> sup E 
«e[o,i] -^0 



{ul-Kt + lY-{ut-ul + Kt-l) 



dt - Ce^ 



and 



E 



thus 



Js r?GU2([s,r]) 

E / (Kf - 1)2 dt < Ce^. 

JO 

We denote *B = t) G J] x [0, 1] : h + > -Ce^} and define 

f if fct + ^-f > -Cei 
I dr^l otherwise. 

Hence a simple calculation shows that 



(4.16) 



E [\kt + ^t)d iv - V')t = E [\kt + n)x<Bd {-V')t > -Ce-^: 
Jo Jo 

E [\kt + ^)x<Bdvl<Ce' 
Jo 



which implies that 



(4.17) 



It worth montioning that the above conditions reveals the minimum qual- 
ification for the pair (x^, (u^, r/^)) to be e— optimal. As an example, the 
admissible controls {u^,r]^) = (1 — e2,r/^) are condidate e— optimality, where 
e > is sufficiently small. Note that the first-order adjoint equation is 



d^l = KfdWt 



(4.18) 



with the corresponding trajectories xf = {l—e'2)Wt+vil then the unique solu- 
tion pair of the first-order adjoint equation will be (^'f, Kf) = ^(1 — e^)Wt + 1, (1 — £2 ) 
Hence ([113]) and ([^13]) will be satisfied. 

Conversely, for the sufficient part, since the hamiltonian H (t, x, u, p, q) = u— 
qu is concave in {x,u), we use Theorem 1 to conclude that Mf = (l— (^) )is 
a condidate to be e— optimality for sufficiently small e is indeed an e— optimal 
control. 
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Appendix 



The following result gives some basic properties of the Clarke's gneralized 
gradient. 

Proposition Al. If / : M" M is locally Lipschitz at x G M", then the 
following statements holds 

(1) dxf (x) is nonempty, compact, and convex set in M" 
{2)dA-f) {x) = -dAf) (x), 

(3) € (9a; (/) (x) if / attains a local minimum or maximum at x 

(4) If / is Frechet-differentiable at x, then dxf (x) = {/' {x)} . 

(5) If /, g : ^ M are locally Lipschitz at x G M'^, then 

dx if + g) {x) C dxf {x) + 8x9 {x) ■ 

See (|19] Lemma 2.3), [9J for the detailed proof of the above Proposition. 
As an example, the Clarke's generalized gradient of the absolute value func- 
tion f : X ^ \x\ which is continuously differentiable everywhere except at 
0. Since /' (x) = 1 for x > and, /' (x) = — 1 for x < then the Clarke's 
generalized gradient of / at x = is given by 

5x/(0) = co{-l,l} = [-l,l], 
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